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SYSTEM AND METHOD FOR PROVIDING A SCALABLE 
OBJECTIVE METRIC FOR AUTOMATIC VIDEO QUALITY EVALUATION 
EMPLOYING INTERDEPENDENT OBJECTIVE METRICS 

5 

REFERENCE TO PROVISIONAL APPLICATION 

This patent application refers to and claims the priority and 
benefit of Provisional Patent Application Serial No. 60/260,842 
filed January 10, 2001. 

10 

C3 RELATED APPLICATIONS 

v3 This patent application^ is related to co-pending United States 

'J Patent Application Serial No. 09/734,823 filed December 12, 2000 by 
Ali et al . entitled "System and Method for Providing a Scalable 
i|2 Dynamic Objective Metric for Automatic Video Quality Evaluation." 
ly The present invention is related to that disclosed in United States 

b 

P Patent Application Serial No. 09/817,981 filed March 27, 2001 by 
Ali et al . entitled "System and Method for Optimizing Control 
Parameter Settings in a Chain of Video Processing Algorithms." 
20 Both of the related patent applications are commonly assigned to 
the assignee of the present invention. The disclosures of both of 
the related patent applications are hereby incorporated by 
reference in the present application as if fully set forth herein. 



25 
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TECHNICAL FIELD OF THE INVENTION 

The present invention is generally directed to systems and 
methods for evaluating video quality, and, in particular, to an 
improved system and method for providing a scalable objective 
5 metric employing interdependent objective metrics for automatically 
evaluating video quality of a video image. 



^'Z BACKGROUND OF THE INVENTION 

Video experts continually seek new algorithms and methods for 

ru 

iQg improving the quality of video images. The primary goal is to 
obtain the most perceptually appealing video image possible. The 

3 

C3 ultimate criterion is the question "How well does the viewer like 

m 

W the resulting picture?" One way to answer the question is to have 

Q 
I'll 

a panel of viewers watch certain video sequences and then record 
15 the opinions of the viewers concerning the resulting image quality. 



The results, however, will vary from panel to panel according to 



the variability between the viewing panels. This problem is 



commonly encountered when relying on subjective human opinion. The 



severity of the problem is increased when the viewing panel is 



20 composed of non-experts. 



Results solely based upon on human perception and subjective 
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opinion are usually subjected to subsequent statistical analysis to 
remove ambiguities that result from the non-deterministic nature of 
subjective results. Linear and non-linear heuristic statistical 
models have been proposed to normalize these types of subjective 
5 results and obtain certain figures of merit that represent the 
goodness^ (or the degradation) of video quality. The process of 
measuring video quality in this manner is referred to as 
2 "subjective video quality assessment." 

Subjective video quality assessment methods give valid 

fU ' ■ ■ ^ 

log indications of visible video artifacts. Subjective video quality 

i 

assessment methods, however, are probabilistic in nature, complex, 

P time consuming, and sometimes difficult to apply. In addition, 

CO 

W there is a problem in selecting appropriate viewers for the viewing 

Q 

panel. A non-trained viewer will be a poor judge of the 

CSS 

15 suitability of new video processing methods. A non-trained viewer, 
however, will likely accurately represent the general consumers in 
the marketplace. On the other hand, a trained expert viewer will 
be overly biased toward detecting minor defects that will never be 
noticed by the general consumer . 

20 To avoid the disadvantages that attend subjective methods for 

evaluating video quality, it is desirable to use automated 
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objective methods to evaluate video quality. Automated objective 
methods seek to obtain objective figure of merits to quantify the 
goodness (or the degradation) of video quality. The process for 
obtaining one or more objective measures of the video quality must 
5 be automated in order to be able to quickly analyze differing types 
of video algorithms as the video algorithms sequentially appear in 
a video stream. 

Q Objective measures of video quality are fully deterministic. 

v3 That is, the results will always be the same when the test is 
iiy repeated (assuming the same settings are preserved) . 
""^^ Because the ultimate goal is to present the viewer with the 

most appealing picture, a final judge of the value of the objective 

Cy 

measures of video quality is the degree of correlation that the 

AS. 

U 

i^c objective measures have with the subjective results. Statistical 

few 

B I 
I'- 

15 analysis is usually used to correlate the results objectively 

obtained (automatically generated) with the results subjectively 

obtained (from human opinion) . 

There is a need in the art for improved systems and methods 

for automatically measuring video quality. The process of 
20 automatically measuring video quality is referred to as "objective 

video quality assessment." 
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Several different types of algorithms have been proposed that 
are capable of providing objective video quality assessment. The 
algorithms are generally referred to as "objective video quality 
models." A report from the Video Quality Experts Group (VQEG) sets 
forth and describes the results of an evaluation performed on ten 
(10) objective video quality models. The report is dated December 
1999 and is entitled "Final Report from the Video Quality Experts 
Group on the validation of Objective Models of Video Quality 
Assessment." The report is presently available on the World Wide 
Web at http : //www -ext . crc . ca/VQEG. 

Each different objective video quality model provides its own 
distinctive measurement of video quality referred to as an 
"objective metric." A "double ended" objective metric is one that 
evaluates video quality using a first original video image and a 
second processed video image. A "double ended" objective metric 
compares the first original video image to the second processed 
video image to evaluate video quality by determining changes in the 
original video image. A "single ended" objective metric is one 
that evaluates video quality without referring to the original 
video image. A "single ended" objective metric applies an 
algorithm to a video image to evaluate its quality. 
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No single objective metric has been found to be superior to 
all the other objective metrics under all conditions and for all 
video artifacts. Each objective metric has its own advantages and 
disadvantages. Objective metrics differ widely in performance 
5 (i.e., how well their results correlate with subjective quality 
assessment results), and in stability (i.e., how well they handle 
different types of video artifacts), and in complexity (i.e., how 

Q much computation power is needed to perform the algorithm 

I 

^^g calculations) . 

i|U A wide range of applications exists to which objective metrics 

?=> 

\U may be applied. For example, fast real-time objective metrics are 
needed to judge the quality of a broadcast video signal. On the 

Q 

other hand, more complex and reliable objective metrics are better 

□ 

p for judging the quality of non-real time video simulations. 

15 Using only one objective metric (and one objective video 

quality model) limits the evaluation of the quality of a video 
signal to the level of evaluation that is obtainable from the 
objective metric that is used. It is therefore desirable to use 
more than one objective metric for video quality evaluation. 

2 0 An improved system and method that uses more than one objective 
metric for video quality evaluation has been disclosed in United 
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States Patent Application Serial No. 09/734,823 filed December 12, 
2000 by Ali et al . entitled "System and Method for Providing a 
Scalable Dynamic Objective Metric for Automatic Video Quality 
Evaluation. " 

There is a need in the art for an improved system and method 
for combining objective metrics in order to form more efficient 
objective metrics for video quality evaluation. 
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SUMMARY OF THE INVENTION 



The present invention generally comprises an improved system 
and method for providing a scalable objective metric employing 
5 interdependent objective metrics for automatically evaluating video 
quality of a video image. 

In an advantageous embodiment of the present invention, the 
Q improved system of the invention comprises an objective metric 
%Q controller that is capable of receiving a plurality of objective 
i|y metric figures of merit from a plurality of objective metric model 
units. The objective metric controller is capable of using 
objective metrics for both desirable and undesirable video image 

CO 

> s characteristics. The objective metric controller is also capable 

iy 

Q 

Q of using a plurality of interdependent objective metrics. The 
15 objective metric controller is capable of determining a scalable 
objective metric from the plurality of interdependent objective 
figures of merit. 

In an advantageous embodiment of the present invention, the 
improved method of the invention comprises the steps of 
20 (1) receiving in an objective metric controller a plurality of 
objective metric figures of merit from a plurality of objective 
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metric model units comprising at least one pair of objective metric 
model units that is interdependent, and (2) determining a scalable 
objective metric from the plurality of the objective metric figures 
of merit. 

5 It is a primary object of the present invention to provide an 

improved system and method for providing a scalable objective 
metric for automatically evaluating video quality of a video image 

%.0 using interdependent object metric model units. 

It is another object of the present invention to provide a 

ry 

idj scalable objective metric from a correlation factor derived from a 

SSI 

mathematical description of an interdependency of at least one 

p 

interdependent pair of objective metric model units. 

It is an additional object of the present invention to provide 

ci 

^ a scalable objective metric from correlation factor derived using a 
15 neural network algorithm that employs both objective quality scores 
and subjective quality scores. 

It is another object of the present invention to continually 
determine new values of the scalable objective metric from new 
values of the plurality of objective metric figures of merit as new 
20 video images are continually received. 

The foregoing has outlined rather broadly the features and 
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technical advantages of the present invention so that those skilled 
in the art may better understand the Detailed Description of the 
Invention that follows. Additional features and advantages of the 
invention will be described hereinafter that form the subject of 
5 the claims of the invention. Those skilled in the art should 
appreciate that they may readily use the conception and the 
specific embodiment disclosed as a basis for modifying or designing 

Q other structures for carrying out the same purposes of the present 

. 

invention. Those skilled in the art should also realize that such 
10^ equivalent constructions do not depart from the spirit and scope of 

15. 

the invention in its broadest form, 
p Before undertaking the Detailed Description of the Invention, 

it may be advantageous to set forth definitions of certain words 

Q 

tj and phrases used throughout this patent document : the terms 

s 

15 "include" and "comprise" and derivatives thereof, mean inclusion 
without limitation; the term "or," is inclusive, meaning and/or; 
the phrases "associated with" and "associated therewith," as well 
as derivatives thereof, may mean to include, be included within, 
interconnect with, contain, be contained within, connect to or 

20 with, couple to or with, be communicable with, cooperate with, 
interleave, juxtapose, be proximate to, be bound to or with, have. 
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have a property of, or the like; and the term "controller," 
"processor," or "apparatus" means any device, system or part 
thereof that controls at least one operation, such a device may be 
implemented in hardware, firmware or software, or some combination 
of at least two of the same. It should be noted that the 
functionality associated with any particular controller may be 
centralized or distributed, whether locally or remotely. 
Definitions for certain words and phrases are provided throughout 
this patent document, those of ordinary skill in the art should 
understand that in many, if not most instances, such definitions 
apply to prior, as well as future uses of such defined words and 
phrases . 
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BRIEF DESCRIPTION OF THE DRAWINGS 

For a more complete understanding of the present invention, 
and the advantages thereof, reference is now made to the following 
5 descriptions taken in conjunction with the accompanying drawings, 
wherein like numbers designate like objects, and in which: 

FIGURE 1 is a block diagram that illustrates (1) a plurality 
Q of objective metric model units for obtaining a plurality of 
^3 objective metric figures of merit from a video stream and 
10^ (2) a objective metric controller capable of using the plurality of 

asp 

'5 objective metric figures of merit to determine a scalable objective 
metric; 

[y FIGURE 2 is a flow chart diagram illustrating an advantageous 

P 

O method of using a plurality of objective metric figures of merit to 
15 determine a scalable objective metric; 

FIGURE 3 is a flow chart diagram illustrating an advantageous 
method of operation of the improved system and method of the 
present invention; and 

FIGURE 4 is a flow chart diagram illustrating an alternative 
20 advantageous method of operation of the improved system and method 
of the present invention. 
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DETAILED DESCRIPTION OF THE INVENTION 

FIGURES 1 through 4, discussed below, and the various 
embodiments set forth in this patent document to describe the 
5 principles of the improved system and method of the present 
invention are by way of illustration only and should not be 
construed in any way to limit the scope of the invention. Those 
skilled in the art will readily understand that the principles of 

%U 

p the present invention may also be successfully applied in any type 

ess ^ 

ry 

10,2 of device for evaluating video quality. 

FIGURE 1 illustrates system 100 for providing a scalable 

P objective metric for automatic video quality evaluation. System 100 

CO 

receives video stream 110. Each of a plurality of objective metric 

Q 

P model units (120, 130, ... , 140) receives a copy of the video 
15 signal of video stream 110. Objective metric model unit 120 applies 
a first objective metric model (referred to as "Metric 1") to 
obtain a first figure of merit, f(l), that represents the quality 
of the video signal based on the first objective metric model. The 
first figure of merit, fd), is provided to controller 150. 
20 Similarly, objective metric model unit 130 applies a second 

objective metric model (referred to as "Metric 2") to obtain a 
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second figure of merit, f(2), that represents the quality of the 
video signal based on the second objective metric model. The 
second figure of merit, f(2), is also provided to controller 150. 
Continuing in this manner, other objective metric model units are 
5 added until the last objective metric model unit 140 has been 
added. Objective metric model unit 140 applies the last objective 
metric model (referred to as "Metric N" ) . Objective metric model 

C3 

%U units (120, 130, 140) obtain a plurality of figures of merit 

»F (f(l), f(2), f (N) ) and provide them to controller 150. 

ry 

The figures of merit (f(l), f(2), f (N) ) represent a 

^'^ series of N evaluations of the quality of the video stream by 

a 

N different objective metrics. The figures of merit (f(l), f(2), 
P f (N) ) may also be designated f(i) where the value of i goes 

1^ from 1 to N. 

15 As will be explained below in greater detail, system 100 of 

the present invention provides a system and method for using the 
figures of merit f(i) to calculate a scalable objective metric. 
The letter "F" (shown in FIGURE 1) designates the scalable 
objective metric of the present invention. 

20 System 100 of the present invention comprises controller 150 

and memory 160. Controller 150 may comprise a conventional 
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microprocessor chip or specially designed hardware. Controller 150 
is coupled to a plurality of objective metric model units (120, 
130, ... , 140) via signal communication lines (shown in FIGURE 1) . 
Controller 150 operates in conjunction with an operating system 
5 (not shown) located within memory 160 to process data, to store 
data, to retrieve data and to output data. Controller 150 
calculates scalable objective metric "F" by executing computer 

%J 

instructions stored in memory 160. 

v3 

Memory 160 may comprise random access memory (RAM) , read only 

fy 

iC memory (ROM) , or a combination of random access memory (RAM) and 
read only memory (ROM) . In an advantageous embodiment of the 
IQ present invention, memory 160 may comprise a non-volatile random 

ly 

p access memory (RAM) , such as flash memory. Memory 160 may also 

C3 

1^ comprise a mass storage data device, such as a hard disk drive 
15 (not shown in FIGURE 1) or a compact disk read only memory (CD-ROM) 
(not shown in FIGURE 1) . 

It is noted that the system and method of the present 
invention may be used in a wide variety of types of video 
processing systems, including, without limitation, hard disk drive 
20 based television sets and hard disk drive based video recorders, 
such as a ReplayTV™ video recorder or a TiVO™ video recorder. 
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Controller 150 and metric calculation algorithm 170 together 
comprise an objective metric controller that is capable of carrying 
out the present invention. Under the direction of computer 
instructions in metric calculation algorithm 170 stored within 
5 memory 160, controller 150 calculates a scalable objective metric 
"F" using the figures of merit f (i) . 

A weighting unit 190 within controller 150 dynamically detects 
P the currently occurring characteristics of the video sequence. The 
%ij currently occurring characteristics may include such features as 

fll 

loZ sharpness, color, saturation, motion, and similar types of 

sis 

33, 

features. Weighting unit 190 assigns a value (or "weight") w(i) to 
n each objective metric (Metric 1, Metric 2, Metric N) . For 

■551? 

lU 

ly example, if Metric 1 is especially good when used on a certain 

n 

Q first type of video signal, then the value of w(l) is given a 
15 greater value than the other values of w(i) . Conversely, if Metric 
2 is not very good when used on that same first type of video 
signal, then w(2) will be given a lower value than the other values 
of w(i) . If a second type of video signal is present, it may be 
that Metric 1 is not as good as Metric 2 when used on the second 
20 type of video signal. In that case, w(2) is given a higher value 
and w(l) is given a lower value than the other values of w(i) . 
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Generally speaking, the values of w{i) that weighting unit 190 
selects will vary depending upon the type of video signal that 
weighing unit 190 dynamically detects. Controller 150 uses metric 
calculation algorithm 170 to calculate the sum S of the products of 
5 each w(i) and f(i). That is, 



C3 



S = w(l)f(l) + w(2)f(2) + ... + w(N)f(N) (1) 



or S ^ w(i)f(i) (2) 



r= where the value of i runs from 1 to N. 

P A correlation factor r(i) is associated with each figure of 

ill merit f (i) . The correlation factor r(i) is obtained from the 

P 

P expression: 

15 r(i) = 1 - [ A(i) / B ] (3) 

where 

A(i) = 6 £ [(X(i,j) - Y(i,j)]' (4) 
where the value of j runs from 1 to n. 
and where 

20 B = n (n^ - 1) (5) 
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The values of X(i,j) are the values of a set of n objective data 
values for a video image. The values of Y(i,j) are the values of a 
set of n subjective data values for the same video image. That is, 
the number of X data points (n) is the same number of Y data points 
5 (n) . 

The value r(i) is referred to as the "Spearman rank" 
correlation factor. The value r(i) is a measure of how well the 
C3 objective X values match the subjective Y values. The values of 
%Q the correlations factors r(i) for each figure of merit f(i) are 
ijy known, having been previously determined by statistical analysis. 
'"2 Values of the correlation factors r(i) are stored in metric 

ah 

p parameters look up table 180 in memory 150. 

|y A "best fitting" value for scalable objective metric "F" is 

ci 

P desired. The "best fitting" value of "F" represents the highest 
15 level of correlation of the objective metric measurements of video 
quality (generated automatically) and the subjective measurements 
of video quality (from human opinions) . The "best fitting" value of 
"F" represents the closest approximation of the subjective 
measurements of video quality by the objective measurements of 
20 video quality. Because the video images in a video stream are 
constantly changing, the "best fit" will require constant automatic 
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updating. The term "dynamic" refers to the ability of the 
objective metric of the present invention to continually change its 
value to take into account the continual changes of the video 
images in a video stream. 

5 As previously mentioned, weighting unit 190 continually (i.e., 

dynamically) detects the characteristics of the video sequence as 
they occur. For each correlation factor r(i), weighting unit 190 

O continually assigns values of w(i) to each figure of merit f{i). 
To dynamically obtain the "best fitting" value of "F" , metric 
loH calculation algorithm 170 determines the values of w(i) that cause 

■S the value S to be a maximum for each value of r(i) . The largest of 

a 

Q these values (i.e., the maximum value) is selected to be the 

CO 

Id scalable objective metric "F." That is, 

ca 
□ 

15 F = Maximum [ S(r{l)), S(r(2)), ... , S(r(N)) ] (6) 

Scalable objective metric "F" is referred to as "scalable" 
because new objective metric model units can be easily added (as 
long as their correlation factors r(i) are defined). In addition, 
20 objective metric model units that are no longer desired can easily 
be removed. 
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The scalable objective metric "F" of the present invention 
provides a great deal of flexibility. For example, for fast (real 
time) video signals, any complicated measurement objective metrics 
may be switched off so that their figures of merit are not 
considered in the metric calculation process. For simulation and 
video chain optimization applications, where more time can be used 
to perform the metric calculation, the more complicated measurement 
objective metrics may be switched on so that their figures of merit 
may be considered in the metric calculation process , 

The scalable objective metric of the present invention avoids 
the shortcomings of any single objective metric. This is because 
weighting unit 190 will assign a low value to w(i) for any 
objective metric that performs poorly in the presence of a certain 
set of artifacts. The scalable objective metric of the present 
invention achieves the highest correlation with the results of 
subjective testing when compared any single objective metric. The 
scalable objective metric of the present invention will be at least 
as good as the best single objective metric under all 
circumstances. Because the scalable objective metric permits the 
inclusion of any objective metric, the system and method of the 
present invention is not limited to use with a particular type of 
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objective metric (e.g., a "single ended" objective metric or a 
"double ended" objective metric) . 

It is noted that the elements of the present invention that 
have been implemented in software (e.g., weighting unit 190) may be 
5 implemented in hardware if so desired. 

As shown in FIGURE 1, system 100 of the present invention also 
comprises neural network unit 195. In one embodiment neural 

O 

^ network unit 195 may be located within controller 150. The 

sU 

operation of neural network unit 195 will be more fully described 

ry 

iQc below. 

S FIGURE 2 is a flow chart diagram illustrating the method of 

W operation of the system of the present invention. The steps of the 

CO 

method are generally denoted with reference numeral 200. A video 

Q 

^'r' image from video stream 110 is provided to N objective metric model 
15 units (120, 130, ... , 140) . The N objective metric model units 
(120, 130, ... , 140) evaluate the video image and obtain N 
respective figures of merit, f(i) (step 205). 

Weighting unit 190 in objective metric controller 150 then 
dynamically detects video characteristics of the video image and 
20 assigns N weights, w(i) , to the N figures of merit, f(i) 
(step 210). For each correlation factor, r(i), objective metric 
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controller 150 calculates a sum, S(r(i)), that is equal to the sum 
of- each product of weight, w(i) , and figure of merit, f(i) (step 
215) . 

Objective metric controller 150 then selects the maximum value 
5 of the sum, S(r(i)), that corresponds to the best correlation of 
objective measurements of video quality with subjective 
measurements of video quality (step 220) . Objective metric 
\Q controller 150 then assigns that value to be the value of the 

=S scalable objective metric "F" (step 225) . Objective metric 

ry 

laP controller 150 then outputs that value of "F" (step 230) . 



After the value of "F" has been output, a determination is 

s 

£33, 

made whether objective metric controller 150 is still receiving 

id 

p video images (decision step 235) . If the video has ended, then the 

ci 

|j, process ends. If the video has not ended and more video images are 
15 being received, control passes back to step 205 and the objective 
controller 150 continues to operate in the manner that has been 
described. 

The present invention has been described as a system for 
providing a scalable objective metric for evaluating video quality 
20 of a video image. It is understood that the "scalable objective 
metric" of the present invention is a general case that includes as 
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a subset the more specific case of providing "static objective 
metric." To provide a "static objective metric" the present 
invention 1) receives a plurality of objective metric figures of 
merit from a plurality of objective metric model units, and 2) 

5 determines a weight value, w(i) , for each of the plurality of 
objective metric figures of merit, and 3) thereafter keeps the 
weight values, w(i), constant (i.e., unchanged) during the process 

^3 of calculating objective metric "F" for video stream 110. 

The present invention also comprises a system and method for 

fy 

lOp calculating an objective metric "F" by using both single objective 
metrics that represent desired image features and single objective 

3 

p metrics that represent undesired image features. Examples of 

CO 

iU desired image features are sharpness and contrast. Examples of 

9 

C3 undesired image features are noise, blockiness, and aliasing. 
15 A dynamic objective metric "F" that produces good results may be 
obtained by using competing single objective metrics. That is, 
single objective metrics that representing both desired and 
undesired image features are to be combined. 

The single objective metrics may be interdependent. 
20 For example, consider a simple sharpness objective metric that is 
dependent on the presence of noise in the image. Let the sharpness 
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of an image be represented by the signal power Ph in a high 
frequency band Br. Enhancing the sharpness of the image will 
increase the signal power in this frequency band to Ph' where Ph' is 
equal to Ph plus the change in Ph (i.e., APh) . This may be 
expressed as : 

Ph' = Ph + APh (7) 
The measured signal power is an indication of the image sharpness. 

Adding white noise to the clean image will also increase the 
signal power to: 

Ph" = Ph + Nh (8) 
Where Nh is the noise power in frequency band Bh- The sharpness 
metric should therefore be defined as the total signal power Ph 
minus the measured noise power Nh. The sharpness metric is 
interdependent on the noise metric. 

If single objective metrics are used that are not 
interdependent, then scalable objective metric "F" is calculated as 
in Equation (6) . The weight factors that are assigned to desired 
features are given the opposite sign of weight factors that are 
assigned to undesired features. 

If single objective metrics are used that are 
interdependent, then scalable objective metric "F" is not 
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necessarily a linear function of the values of the single objective 
metrics. When interdependent single objective metrics are present 
the value of the scalable objective metric "F" may be determined by 
(1) describing the interdependencies with mathematical equations, 
and (2) correlating the images that correspond to the 
interdependencies with subjective quality scores. 

Alternatively, the value of the scalable objective metric "F" 
may be determined by using a neural network algorithm that employs 
both objective quality scores and subjective quality scores. In 
this embodiment of the invention, controller 150 employs neural 
network unit 195 to calculate a value of the scalable objective 
metric "F" from the values of the interdependent objective metrics. 
In one embodiment, neural network unit 195 is located within 
controller 150. In other embodiments, neural network unit 195 may 
be located externally to controller 150. 

FIGURE 3 is a flow chart diagram illustrating an alternate 
method of operation of the system of the present invention. The 
steps of the method are generally denoted with reference numeral 
300. A video image from video stream 110 is provided to N objective 
metric model units (120, 130, ... , 140). The N objective metric 
model units (120, 130, ... , 140) evaluate the video image and 
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obtain N respective figures of merit, f (i) (step 305) . 

Weighting unit 190 in objective metric controller 150 then 
dynamically detects video characteristics of the video image and 
assigns N weights, w(i), to the N figures of merit, f(i) 
5 (step 310). For independent (i.e., non- interdependent) objective 
metrics, objective metric controller 150 calculates a sum, S(r(i)), 
using a correlation factor, r(i). The sum, S(r(i)), is equal to 
□ the sum of each product of weight, w(i) , and figure of merit, f(i) 
(step 315) ■ 

ess 

lo^y For independent (i.e., non- interdependent ) objective metrics, 

objective metric controller 150 then selects the maximum value of 
the sum, S(r(i)), that corresponds to the best correlation of 

CO 

l^j objective measurements of video quality with subjective 

Q 

P measurements of video quality (step 320) . Objective metric 
15 controller 150 then assigns that value to be the value of the 
scalable objective metric "F" (step 325) . 

For interdependent objective metrics, objective metric 
controller 150 calculates a value of the scalable objective metric 
"F" from a mathematical description of the interdependencies of the 
20 interdependent objective metrics (Step 330) . 

Objective metric controller 150 then outputs the value of "F" 
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(step 335) . After the value of "F" has been output, a determination 
is made whether objective metric controller 150 is still receiving 
video images (decision step 340) . If the video has ended, then the 
process ends. If the video has not ended and more video images are 
being received, control passes back to step 305 and the objective 
controller 150 continues to operate in the manner that has been 
described. 

FIGURE 4 is a flow chart diagram illustrating an alternate 
method of operation of the system of the present invention. The 
steps of the method are generally denoted with reference numeral 
400. A video image from video stream 110 is provided to N objective 
metric model units (120, 130, ... , 140). The N objective metric 
model units (120, 130, ... , 140) evaluate the video image and 
obtain N respective figures of merit, f (i) (step 410) . 

For independent or interdependent objective metrics, objective 
metric controller 150 uses neural network unit 195 to calculate a 
value of the scalable objective metric "F" from the values of the 
interdependent objective metrics (Step 420) . The neural network 
algorithm in neural network unit 195 has previously been trained 
with subjective video quality scores . 

Objective metric controller 150 then outputs the value of "F" 
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(step 430) . After the value of "F" has been output, a determination 
is made whether objective metric controller 150 is still receiving 
video images (decision step 440) . If the video has ended, then the 
process ends. If the video has not ended and more video images are 
5 being received, control passes back to step 410 and the objective 
controller 150 continues to operate in the manner that has been 
described. 

^0 Although the present invention has been described in detail, 

those skilled in the art should understand that they can make 
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